Semantic Robot Programming for Goal-Directed Manipulation in Cluttered Scenes

We present the Semantic Robot Programming (SRP) paradigm as a convergence of robot programming by demonstration and semantic mapping. In SRP, a user can directly program a robot manipulator by demonstrating a snapshot of their intended goal scene in workspace. The robot then parses this goal as a scene graph comprised of object poses and inter-object relations, assuming known object geometries. Task and motion planning is then used to realize the user's goal from an arbitrary initial scene configuration. Even when faced with different initial scene configurations, SRP enables the robot to seamlessly adapt to reach the user's demonstrated goal. For scene perception, we propose the Discriminatively-Informed Generative Estimation of Scenes and Transforms (DIGEST) method to infer the initial and goal states of the world from RGBD images. The efficacy of SRP with DIGEST perception is demonstrated for the task of tray-setting with a Michigan Progress Fetch robot. Scene perception and task execution are evaluated with a public household occlusion dataset and our cluttered scene dataset.Comment: published in ICRA 201

Robotic Manipulation under Transparency and Translucency from Light-field Sensing

From frosted windows to plastic containers to refractive fluids, transparency and translucency are prevalent in human environments. The material properties of translucent objects challenge many of our assumptions in robotic perception. For example, the most common RGB-D sensors require the sensing of an infrared structured pattern from a Lambertian reflectance of surfaces. As such, transparent and translucent objects often remain invisible to robot perception. Thus, introducing methods that would enable robots to correctly perceive and then interact with the environment would be highly beneficial. Light-field (or plenoptic) cameras, for instance, which carry light direction and intensity, make it possible to perceive visual clues on transparent and translucent objects. In this dissertation, we explore the inference of transparent and translucent objects from plenoptic observations for robotic perception and manipulation. We propose a novel plenoptic descriptor, Depth Likelihood Volume (DLV), that incorporates plenoptic observations to represent depth of a pixel as a distribution rather than a single value. Building on the DLV, we present the Plenoptic Monte Carlo Localization algorithm, PMCL, as a generative method to infer 6-DoF poses of objects in settings with translucency. PMCL is able to localize both isolated transparent objects and opaque objects behind translucent objects using a DLV computed from a single view plenoptic observation. The uncertainty induced by transparency and translucency for pose estimation increases greatly as scenes become more cluttered. Under this scenario, we propose GlassLoc to localize feasible grasp poses directly from local DLV features. In GlassLoc, a convolutional neural network is introduced to learn DLV features for classifying grasp poses with grasping confidence. GlassLoc also suppresses the reflectance over multi-view plenoptic observations, which leads to more stable DLV representation. We evaluate GlassLoc in the context of a pick-and-place task for transparent tableware in a cluttered tabletop environment. We further observe that the transparent and translucent objects will generate distinguishable features in the light-field epipolar image plane. With this insight, we propose Light-field Inference of Transparency, LIT, as a two-stage generative-discriminative refractive object localization approach. In the discriminative stage, LIT uses convolutional neural networks to learn reflection and distortion features from photorealistic-rendered light-field images. The learned features guide generative object location inference through local depth estimation and particle optimization. We compare LIT with four state-of-the-art pose estimators to show our efficacy in the transparent object localization task. We perform a robot demonstration by building a champagne tower using the LIT pipeline.PHDRoboticsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/169707/1/zhezhou_1.pd

Multi-species PLIF study of the structures of turbulent premixed methane/air jet flames in the flamelet and thin-reaction zones regimes

Simultaneously planar laser-induced fluorescence (PLIF) measurements of OH, CH, CH2O and toluene are carried out to investigate the structures of turbulent premixed methane/air jet flames in the flamelet regime and the thin-reaction zones regime. A premixed flame jet burner of an inner diameter of 1.5 mm is employed. Stoichiometric methane/air mixtures introduced as a jet are ignited and stabilized in a hot co-flow generated by a coaxial porous plug pilot flame surrounding the jet. The Reynolds number for the studied jet ranges from 960 to 11,500 with the characteristic Karlovitz number ranging from 1 to 60. The focus of this study is on the characterization of the structures and turbulent burning velocity of premixed flames in the flamelet and the thin-reaction zones regimes. The preheat zone is analyzed using the CH2O and toluene PLIF fields, whereas the reaction zone is analyzed using the CH and OH PLIF fields. Laser Doppler Anemometer (LDA) measurements are performed to characterize the turbulence field and it is noted that when the Reynolds/Karlovitz number increases a successive thickening of the preheat zone is observed, whereas the reaction zone, characterized by the CH layer maintains nearly the same thickness. The heat release zone, characterized by the combination of the OH and CH2O PLIF fields, is shown to nearly maintain the same thickness under the present experimental conditions. The flame surface wrinkle ratio is shown to be Reynolds number and Karlovitz number independent when the Reynolds number is high enough such that the smallest wrinkle scales reach to the length scales of the thin reaction layers. The global fuel consumption speed of the jet flame is analyzed using the toluene PLIF field and the OH PLIF field. A discrepancy in the two consumption velocities is found as the Karlovitz number increases. This is found to be a result of the broadening of the oxidation zone. These findings provide experimental support to the flamelet and thin-reaction zone regime hypotheses of turbulent premixed combustion

A 115-bp MethyLight assay for detection of p16 (CDKN2A) methylation as a diagnostic biomarker in human tissues

<p>Abstract</p> <p>Background</p> <p><it>p16 </it>Methylation is a potential biomarker for prediction of malignant transformation of epithelial dysplasia. A probe-based, quantitative, methylation-specific PCR (MSP) called MethyLight may become an eligible method for detecting this marker clinically. We studied oral mucosa biopsies with epithelial dysplasia from 78 patients enrolled in a published 4-years' followup cohort, in which cancer risk for patients with <it>p16 </it>methylation-positive dysplasia was significantly higher than those without <it>p16 </it>methylation (by 150-bp MSP and bisulfite sequencing; +133 ~ +283, transcription starting site, +1). The <it>p16 </it>methylation status in samples (<it>N </it>= 102) containing sufficient DNA was analyzed by the 70-bp classic (+238 ~ +307) and 115-bp novel (+157 ~ +272) MethyLight assays, respectively.</p> <p>Results</p> <p><it>p16 </it>Methylation was detectable in 75 samples using the classic MethyLight assay. The methylated-<it>p16 </it>positive rate and proportion of methylated-<it>p16 </it>by the MethyLight in MSP-positive samples were higher than those in MSP-negative samples (positive rate: 37/44 vs. 38/58, <it>P</it>=0.035, two-sided; proportion [median]: 0.78 vs. 0.02, <it>P <</it>0.007). Using the published results of MSP as a golden standard, we found sensitivity, specificity, and accuracy for this MethyLight assay to be 70.5%, 84.5%, and 55.0%, respectively. Because amplicon of the classic MethyLight procedure only partially overlapped with the MSP amplicon, we further designed a 115-bp novel MethyLight assay in which the amplicon on the sense-strand fully overlapped with the MSP amplicon on the antisense-strand. Using the 115-bp MethyLight assay, we observed methylated-<it>p16 </it>in 26 of 44 MSP-positive samples and 2 of 58 MSP-negative ones (<it>P </it>= 0.000). These results were confirmed with clone sequencing. Sensitivity, specificity, and accuracy using the 115-bp MethyLight assay were 59.1%, 98.3%, and 57.4%, respectively. Significant differences in the oral cancer rate were observed during the followup between patients (≥60 years) with and without methylated-<it>p16 </it>as detected by the 115-bp MethyLight assay (6/8 vs. 6/22, P = 0.034, two-sided).</p> <p>Conclusions</p> <p>The 115-bp MethyLight assay is a useful and practical assay with very high specificity for the detection of <it>p16 </it>methylation clinically.</p